StreamFlow: Cross-Breeding Cloud With HPC
نویسندگان
چکیده
Workflows are among the most commonly used tools in a variety of execution environments. Many them target specific environment; few make it possible to execute an entire workflow different environments, e.g., Kubernetes and batch clusters. We present novel approach execution, called StreamFlow, that complements graph with declarative description potentially complex makes onto multiple sites not sharing common data space. StreamFlow is then exemplified on bioinformatics pipeline for single-cell transcriptomic analysis workflow.
منابع مشابه
Hpc Cloud Applied to Lattice Optimization∗
As Cloud services gain in popularity for enterprise use, vendors are now turning their focus towards providing cloud services suitable for scientific computing. Recently, Amazon Elastic Compute Cloud (EC2) introduced the new Cluster Compute Instances (CCI), a new instance type specifically designed for High Performance Computing (HPC) applications. At Berkeley Lab, the physicists at the Advance...
متن کاملAnalyzing Performance and Efficiency of HPC Applications in the Cloud
Elasticity is a key feature of cloud computing in which the resources can be dynamically changed on-the-fly. In the HPC scenario, a user may want to adjust the resources availability to increase performance. A way to measure performance and efficiency of parallel applications with fixed number of resources is the speedup metric. However this metric does not apply to elastic cloud infrastructure...
متن کاملScheduling Concurrent Workflows in HPC Cloud through Exploiting Schedule Gaps
Many large-scale scientific applications are usually constructed as workflows due to large amounts of interrelated computation and communication. Workflow scheduling has long been a research topic in parallel and distributed computing. However, most previous research focuses on single workflow scheduling. As cloud computing emerges, users can now have easy access to on-demand high performance c...
متن کاملPerformance Evaluation of Multiple Cloud Data Centers Allocations for HPC
This paper evaluates the behavior of the Microsoft Azure G5 cloud instance type over multiple Data Centers. The purpose is to identify if there are major differences between them and to help the users choose the best option for their needs. Our results show that there are differences in the network level for the same instance type in different locations and inside the same location at different...
متن کاملEnergy-Efficient Scheduling of HPC Applications in Cloud Computing Environments
The use of High Performance Computing (HPC) in commercial and consumer IT applications is becoming popular. They need the ability to gain rapid and scalable access to high-end computing capabilities. Cloud computing promises to deliver such a computing infrastructure using data centers so that HPC users can access applications and data from a Cloud anywhere in the world on demand and pay based ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Emerging Topics in Computing
سال: 2021
ISSN: ['2168-6750', '2376-4562']
DOI: https://doi.org/10.1109/tetc.2020.3019202